Entry Name: "UKN-CashCow-MC1"

VAST Challenge 2016
Mini-Challenge 1

Team Members:

Juri Buchmüller, University of Konstanz, Germany, buchmueller@dbvis.inf.uni-konstanz.de, PRIMARY

Manuel Stein, University of Konstanz, Germany, manuel.stein@uni-konstanz.de

Alexander Jäger, University of Konstanz, Germany, alexander.jaeger@uni-konstanz.de

Sabrina Schmidt, Germany, info@passion-fotografie.de

Hansi Senaratne, University of Konstanz, Germany, hansi.senaratne@uni-konstanz.de

Halldór Janetzko, University of Konstanz, Germany, halldor.janetzko@uni-konstanz.de

Student Team: Partly

Tools Used:

MS PowerPoint

Photoshop

Custom control room setup at our labs

Approximately how many hours were spent working on this submission in total?

50 hours.

May we post your submission in the Visual Analytics Benchmark Repository after VAST Challenge 2015 is complete? YES

High-resolution image:

http://files.dbvis.de/get/pheeKigh

Video (Optional but recommended):

http://files.dbvis.de/get/ut7Quivi

Storyboards (Optional but recommended):

http://files.dbvis.de/get/pheeJigh

Description

Provide a description of your design using no greater than 1000 words and 10 images. Describe the important features of your design, including both display features and interactions. Describe how it will enable investigators to quickly understand new situations, think deeply to develop and test their theories as part of their investigation, and rapidly reorient their investigation when data or assumptions change.

We present a task-based interface design for an interactive resort surveillance and investigation system. Besides the three main views for Spatial (4), Entity(3)- and Task-Tracking(5), which are displayed for all operators on three big central wall screens, a core part of our system is the integration of the investigators' direct work environment, which we enhance with ambient light(2) and sound systems(1) that guide the investigators attention. By employing additional clues about the current state by ambient light and sound we are able to increase the situational awareness of the operators.

The ambient sound, much like a car engine, produces a constant ambient noise that does not distract the operators. The noise is generated from a combination of influence factors automatically gathered from resort information: Guest load, on-site crime reports, CCTV imagery, movement behavior models, VIP presence and many more. As situations in the resort arise, e.g. unusually high guest loads, sinking casino revenue, increasing theft/fraud, and so on, the frequencies of the noise changes. Our proposed noise is constructed by overlaying several noise frequencies corresponding to the situational aspects. In normal state, all noise frequencies have the same amplitude. By pre-attentive cognition, operators immediately notice the change in their "engine" and shift their attention and concentration directly to the situation, subconsciously already identifying where the problem lies. Also, they receive further guidance through ambient lights of the screens that guide their attention towards newly emerging elements on the screens.

The screens displaying entities, floor plans, and tasks are visible to all operators at the same time, enabling a collaborative environment, and each operator can pull a view to his touchscreen and work on it. The screen will always be synchronized with the wall screens to ensure transparency between the operators without communication overhead. The color scheme used for the design elements between the screens adheres to the cognitive perception of color in the varying contexts and always remains the same to keep context: VIPs/Entities to be guarded in yellow, dangerous situations, unusual behavior by suspects or known criminals in red, events like conferences in green and general situations like a high guest load in blue.

The leftmost screen displays a list of entities relevant to current operations, annotated with the most important information such as schedules, photos, guest backgrounds, nearest security, etc., about them. As well, interaction possibilities that are specific to the type of entity are provided (e.g. having security monitor / approach suspect), Entities can be persons (e.g. VIPs, or known or detected suspects in the building), events (e.g. conferences, crimes), or situations (e.g. high guest load) that require attention. The list is sorted automatically or manually by an importance ranking of the entity (e.g. a VIP needs to be guarded at all cost, so he is at first place in the ranking). Alerts can connect entities visually, e.g. grouping VIPs and potentially dangerous suspects.

In the middle screen, the main resort areas are displayed. The operators can switch between casino, hotel, and conference area. Symbols next to the areas indicate major events in an area, also for those areas not displayed entirely. These events relate to the entities from the left screen (e.g. VIP present, suspect present, fraud ongoing, etc.). On the floor plans, guests, employees, VIPs, and suspects are color-grouped and can be interactively tracked with movement trajectories. E.g., an operator can request the sight or communicate with security personnel, or detailed information on a person can be accessed. Important or suspicious persons are highlighted. The floor plan also allows interaction with equipment like WiFi-Hotspots and check who is logged in. Heatmaps display aggregated data such as revenue per casino table or guest density, if values differ from expected values in an unusual way.

The right screen shows tasks the operators can define. It is a note-taking environment where tasks can be created from entities, managed, and freely arranged. Here, the operators collect information, define and track tasks, and gather knowledge transparently in the collaborative setting. The view will furthermore be automatically annotated with all available information on the entity, e.g. schedule if known, police records, interaction possibilities, floor position, and more. Orders to employees can directly be issued from the task screen, and the nearest free employee will attend to orders.

We consider that a facility like Eubia Resort is too complex to be monitored by just looking at guests and/or reading sensors. Therefore, an automatic threat and event model enhances the cognition of the operators, being able to hint at developing situations or showing unsuspected connections. The model automatically detects unusual situations with a broad spectrum by fusing basic, measurable data, e.g. changes in revenue, guest density, movement direction of guests, event schedules, CCTV and many more. If the operators detect other events, these can be added into the system by just adding a task. The system then helps tracking all relevant entities. Also, the ambient sound and light system feeds from the threat model and improves the alertness of the operators. As the sound is generated directly from unfiltered incoming data, with enough experience, the operators can even hear and see a certain critical situation build up before the threat model would be certain to display an entity, and thus take preventive action. Aiding the threat model as well is the link to police databases so that known suspects can be identified easily.

With the ability to see what is going on in the premises, to be notified of developing situations, and to collectively share knowledge about tasks, operators can verify their hypotheses quickly, but can also keep track over situations for longer periods of time without forgetting about them. To illustrate these processes better, our storyboard tells the tale of a VIP seemingly being threatened and a Mediterranean finance minister trying to improve his country’s financial situation with shady methods.